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Technical  Information 


1.  Technical  Progress  /  Highlights  -  Observations 

We  have  completed  integrating  the  themed  pre-  and  post-  tests  into  Refraction,  Treefrog  Treasure,  and 
Creature  Capture.  The  tests  are  randomized  in  a  way  that  answers  are  not  the  same  spatially  across  each 
test  (to  use  a  comparison  question  as  an  example,  if  the  student  needs  to  place  <,  >,  <,  and  =  in  the  upper 
left,  lower  left,  upper  right,  and  lower  right,  in  Version  1  of  the  test,  in  Version  2  does  not  echo  the  same 
placement  of  the  symbols  with  new  numbers,  in  case  the  student  remembered  the  order  of  the  answers. 

We  trained  people  from  both  the  Learning  Science  and  Computer  Science  teams  on  the  behavioral  and 
affective  data  collection  protocol  created  by  Ryan  Baker  from  Columbia.  We  are  now  starting  the  trials  at 
University  Child  Development  School  (UCDS)  and  Interagency  Academy  (part  of  Seattle  Public 
Schools),  collecting  this  data  as  well  as  our  standard  logging  of  gameplay  actions. 

Use  of  the  games  in  school  settings  has  already  surfaced  numerous  small  bugs  and  polish  items  for  the 
games;  we  are  reacting  to  this  feedback  immediately  and,  in  some  cases,  turning  around  a  new  version  of 
the  game  for  the  next  day  of  classes  (while  still  taking  care  not  to  alter  the  math  content  of  the  games). 
Some  of  those  points  are  detailed  in  the  section  below. 

We  also  received  much  positive  feedback  about  the  way  the  games  were  received  by  the  students,  even  at 
this  early  stage.  Here  is  a  field  report  with  some  highlights  from  Treefrog  Treasure: 

Oh,  and  overall,  kids  LOVED  the  game.  And  not  just  the  elementary  kids.  I  was  with  the 
alternative  school  kids  this  afternoon  (middle  and  high  school  aged).  A  couple  quotes: 

•  One  kids  says,  "that  was  kinda  fun  and  I  never  have  fun  in  that  class" 

•  One  kids  asked  what  we  were  up  to  and  another  teacher  nearby  said,  "learning 
games"  and  boy  3  said,  "they're  hella  fun". 

•  Another  kid  yells  out,  "Can  we  do  this  everyday  Mr.  McCracken?"  Most  of  the 
rest  of  us  laugh. 

•  There  were  also  a  lot  of  "This  is  crackin'"  comments,  one  kid  compared  it  to 
Sonic  (in  a  good  way),  and  another  kid  wants  it  on  his  360. 

We  interviewed  numerous  candidates  for  the  Community  Manager  position  and  have  narrowed  it  down  to 
two  final  candidates.  We  also  completed  the  request  for  opening  the  Math  Content  and  Experience  Tester 
position  as  described  in  our  communications. 

Preliminary  design  work  and  planning  was  done  this  month  towards  an  investigation  of  growth  mindset  as 
relates  to  our  games.  The  growth  mindset  is  defined  as  a  mindset  where  students  see  intelligence  not  as  a 
fixed  quality  they  are  born  with,  but  rather  a  quality  that  they  can  grow  through  effort  and  hard  work.  In 
particular,  the  growth  mindset  teaches  that  the  brain  is  a  “muscle”  that  one  can  make  stronger  through 
mental  exercise.  To  study  whether  we  can  influence  player’s  mindsets  about  learning  through  our  games, 
we  are  looking  at  developing  and  studying  different  feedback  and  reward  structures. 

2.  Results  or  Problems  and  Solutions 

We  have  begun  to  make  adjustments  to  the  games  based  on  the  play  we  have  seen  in  the  classroom.  Two 
important  examples  of  this  are: 

•  Adjustments  to  progression  length  based  on  average  time  to  complete  the  level  packs  -  As  the 
schools  give  us  certain  windows  of  time  to  let  the  students  play  the  games,  an  important  part  of 
our  adjustment  is  finding  the  right  amount  of  gameplay  to  give  to  the  kids.  Different  kids 


complete  the  game  at  wildly  different  speeds,  so  in  order  to  ensure  that  the  kids  get  to  try  all  of 
the  games,  we  have  reduced  the  number  of  levels  in  certain  games.  We  do  not  believe  this  will 
adversely  affect  learning  as  the  levels  eliminated  were  “extra”  levels  designed  to  pad  out  the 
experience  and  keep  in  engaging  in  between  assessments.  The  experience  may  be  a  little  more 
test-heavy  as  a  result,  but  it  will  also  be  more  interesting  over  a  number  of  days,  as  a  greater 
number  of  games  will  be  played. 

•  Development  of  measures  to  find  students  who  did  not  take  the  assessment  sections  seriously  - 
There  were  some  reports  of  students  simply  clicking  past  the  assessments  without  actually 
“doing”  them.  We  believe  we  will  be  able  to  spot  this  in  the  logging  data  (the  test  will  have  been 
completed  in  mere  seconds).  Additionally,  since  the  students  were  observed,  the  behavioral  and 
affective  data  will  show  the  students  as  “off  task”  during  the  period  the  test  was  present  if  they 
did  not  take  it  seriously. 

3.  Significant  Accomplishments  Anticipated  During  Next  Reporting  Period 

We  will  continue  in-classroom  data  collection  with  pre-  and  post-tests  and  gameplay  data  associated  with 
behavioral  and  affective  data  will  begin.  Feedback  on  problems,  bugs,  or  game  design  issues  from  the 
classroom  will  be  immediately  incorporated  into  the  games  (in  some  cases  for  the  next  day),  allowing  for 
rapid  iteration  on  the  game’s  design  and  implementation  details. 

4.  Publications  relevant  to  this  effort 

Educated  Play!  -  Refraction.  In  Game  Developer  magazine,  October  2012. 

More  than  child’s  play:  Games  have  potential  learning  and  assessment  tools, 
http://www.kappanmagazine.Org/content/94/2/26.abstract,  Vicki  Phillips  and  Zoran  Popovic,  Phi  Delta 
Kappan  October  2012  vol.  94  no.  2  26-30 

New  Game  on  GameUp:  Center  for  Game  Science  presents  “Treefrog  Treasure”.  On  BrainPOP, 

November  30th,  2012. 

5.  Meetings  and  Events  (Please  include  meetings  with  subcontractors) 

November  1 1,  2012.  Prof.  Dan  Schwartz  and  his  student  Nicole  Hallinen  visited  UW  to  discuss  their 
potential  involvement  in  the  project,  such  as  contrasting  cases  as  a  new  game  paradigm,  what  we  can 
apply  it  to,  how  to  construct  experimental  trials,  and  how  we  will  automatically  generate  non-verbal 
contrasting  cases  learning  progressions  from  a  defined  description  of  specific  domain  knowledge. 

November  16,  2012.  “The  Crossroads  of  Engagement  and  Mastery  in  STEM  Education.”  Washington 
STEM  Breakfast  Series,  Seattle,  WA. 

6.  Changes  to  the  Contract  Organization 

As  we  have  been  discussing  in  previous  reports  and  other  communication,  the  UTA  subcontract  will  be 
replaced  by  internal  UW  hiring  and  with  external  consultants  from  Stanford  University.  We  have 
provided  what  we  believe  is  all  of  the  requested  information  regarding  this  change  as  of  November  15th. 


